processing

• comprehension of serially presented written or oral language input in context

incrementality

• build a syntactic & semantic representations as the sentence comes in

  • how to deal with ambiguity? singular guess or parallel hypotheses?

predicitive

• minimal sense: processing behavior is a function of current state

• strong(est) sense: comprehender entertains hypotheses about the future

• basic premiss: anticipation is an advantage
• (maximally) predictive interpreter has lexico-syntactic expecations \(P(w_1, \dots, w_n \mid c)\)
  • operationalized by corpus frequencies of relevant structures
  • possible beam-search approximation
  • derived expectation about continuation \(P(w_{i+1}, \dots, w_n \mid w_1, \dots, w_i, c)\)

• processing difficulty linked to distance between \(P(\cdot \mid w_1, \dots, w_\red{i}, c)\) and \(P(\cdot \mid w_1, \dots, w_\red{i+1}, c)\)
  • self-paced reading times (Smith & Levy 2013)
  • various ERP components, notably N400 amplitude (Frank et al. 2015)

literal listener picks literal interpretation (uniformly at random):

\[ P_{LL}(t \mid m) \propto P(t \mid [\![m]\!]) \]

Gricean speaker approximates informativity-maximization:

\[ P_{S}(m \mid t) \propto \exp( \lambda P_{LL}(t \mid m)) \]

pragmatic listener uses Bayes' rule to infer likely world states:

\[ P_L(t \mid m ) \propto P(t) \cdot P_S(m \mid t) \]

interpretation as holistic: full & complete utterance

• messages are word sequences: \(\messg = w_1, \dots, w_n\)

• initial subsequence of \(\messg\): \(\messg_{\rightarrow i} = w_1, \dots w_i\)

• all messages sharing initial subsequence: \(\Messgs(\messg_{\rightarrow i}) = \set{\messg' \in \Messgs \mid \messg'_{\rightarrow i} = \messg_{\rightarrow i}}\)

• next-word expectation:

\[P_L(w_{i+1} \mid \messg_{\rightarrow i}) \propto \sum_{\state} P(\state) \ \sum_{\messg' \in \Messgs(\messg_{\rightarrow i}, w_{i+1})} P_S(\messg' \mid \state)\]

• interpretation evidence:

\[P_L(\state \mid \messg_{\rightarrow i}) \propto P(\state) \ \sum_{\messg' \in \Messgs(\messg_{\rightarrow i})} P_S(\messg' \mid \state)\]

participants & procedure

• EEG recording of 25 native German speakers
• picture (1500ms) -> sentence (500ms per word) -> truth-value judgement

sentence material

• "Alle/Einige\(_1\) Punkte sind blau\(_2\), die im Kreis/Quadrat\(_3\) sind"
• "All/some of the dots in the circle/square are blue/red"

visual stimuli

general assumptions

• listener expects speaker to make a pragmatically felicitous utterance
• listener does not give up on speaker rationality on the way (charity, forward induction, …)

experimental microcosmos assumption

• possible meanings \(\States\): pairs of contexts (\(A\) - \(D\)) and speaker-intended shape
• possible messages \(\Messgs\): "All/some dots are blue/red that are in the square/circle."

specific assumptions

• listener knows context, but not shape
• speaker chooses description for context and shape

• incremental RSA predicts \(P_L(w_{i+1} \mid w_{1,\dots,i}, c)\)
• correlating predicted next-word expectations grand-average early N400 (300-400ms):
  • \(r= 0.44\), \(p < 0.01\) in total
  • \(r = 0.81\), \(p < 0.001\) after exclusion of unexpected continuations

pragmatic responders

expect pragmatic speakers

\[P(\text{more informative true}) > P(\text{less informative true}) > P(\text{false})\]

semantic responders

expect literal speakers

\[P(\text{more informative true}) = P(\text{less informative true}) > P(\text{false})\]

• correlation between predictions and observations (excluding redundant relative clauses):
  • \(r= 0.69\), \(p < 0.01\) for pragmatic responders
  • \(r = 0.57\), \(p < 0.01\) for semantic responders

main issue

how to fix reasonable \(\States\) and \(\Messgs\)?

experimental microcosmos assumption

all (and only?) meanings and forms that occur in the experiment

prediction

massive influence of filler material

participants & procedure

• EEG recording of 24 native German speakers
• picture (1500ms) -> sentence (500ms per word) -> truth-value judgement

sentence material

• "Einige\(_1\) Punkte sind blau\(_2\), die im Kreis/Quadrat\(_3\) sind"
• "Einige\(_1\) Punkte sind blau\(_2\)"

visual stimuli

behavioral data

only one participant consistently gave pragmatic judgements

ERP responses

no trace of pragmatic infelicity / expectations

• incremental RSA seems feasible:
  • next-word expecations & accummulated interpretation evidence
  • genuine pragmatic expectations beyond / on top of lexico-pragmatic expectations

• main challenges:
  • link functions for interesting experimental measures
  • how to fix \(\States\) and \(\Messgs\) in general?
  • how to scale up to more open-ended applications? tradeoff lexico-syntactic vs. pragmatic expectations?